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ABSTRACT 

This docunent introduces a study conducted! by the 
National Council of social Studies (NCSS) to review, 1; tetbret, and 
disseminate findings of the National Assessment of Educatilnal 
Progress (NAEP) assessments in citizenship and social studies. The 
study represents an investigation funded by the Education (pmmission 
of the states and coordinated by a special Steering Commit-i|se working 
under the auspices of NCSS. Six tasks were identified by thf 
committee to examine NAEP; methods and procedures; the valillity of 
the exercise; interpretation of the findings; consistency o| the 
exercises with NCSS Social Studies Curriculum Guidelines, 1 
desirability, and realistically satisfactory performance le^|els; the 
assessment model; and dissemination. Also included in this document 
is background information on the NAEP, the purpose of which was to 
make available to those interested in education the attainments of 
students in citizenship and social studies. A major portion of this 
document focuses on NAEP decisions on a host of issues; for example, 
in considering objectives, who was to decide what was to be assessed 
in social studies and, moreover, what the basic criteria should be 
for the objectives. The latter portion of the document points out 
that the NAEP assessment in itself is neither an evaluation nor 
explanation, but is a collection of data provided in national 
percentages for performance of each exercise, by age, regions, sex, 
race, parental education, and size and type of community. (DE) 
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INTRODUCTION 

National Assessment and Social Studies 
Education: The Setting 

Jean Fair 

In the midst of unsettling social chan^Je education becomes 
inevitably a matter of public debate and policy niaking. The 
heatr even outright turmoil, of recent years has subsided j and 
faith in education as the road to salvation has given way. Still 
firm is the belief that education is basic to individual and 
social welfare. And still widespread is the uneasy feeling that 
schools are not doing what they ought to be doing, mnd not even 
doing well what they have long been doing. When too little 
money is at hand for public services, the debate is further 
sharpened. 

The pressures in policy making make plain the need for 
information. Simply stated, young people go to school to learn 
something. Although the country has accumulated information on 
scores of no doubt useful matters, little systematic evidence 
has been available on the crucial point of it all, actual , 
educational attainment. 

Efforts to gather evidence soon confront the basic 
questions of the debates what should young people be learning, 
what should they achieve; and to what extent are they doing so? ^ 
To these two are added others: how to find out, and how to 
explain what is found in some way useful to decision-making. 
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Information gathering is no simple task: indeed, it is a 
subject of debate in itself. 

A major effort has been under way by the iJational Assess- 
ment of Educational Progress (NAEP) . Jf special significance 
to social studies education are its recent reports of findings 
in Citizenship and Social Studies, 1969-70 and 1971-72 assessments. 

Some few words of background are needed here. Formal dis- 
cussion of the possibilities of a national assessment of educa- 
tional attainments began some ten years ago. Supported by the 
Carnegie Corporation of New York, a private foundation, the 
Exploratory Committee on Assessing the Progress of Education came 
into being in 1964 to develop a concrete plan. This Committee's 
work became the basis of the present national project. In mid- 
1969 the Educational Commission of the States, a compact pre- 
sently of some 47 states and territories to consider and coordi- 
nate educational efforts and problems, assumed the governance of 
the project. 

The prime purpose of the National Assessment of Educational 
Progress is to make information available to those interested in 
education. Assessments are carried on at regular intervals, at 
the present in ten areas s reading, writing, science, mathematics, 
social studies, citizenship, career and occupational develop- 
ment, literature, art, and music: in short, in much more than 
the 3 R's. NAEP assesses educational achievement, not merely 
school achievement. Obviously schools have responsibilities for 
education. Still television, magazines, libraries, newspapers, 
civic organii;ations, especially those for young people, religious 
institutions, personal opportunities and experiences— these and 



3 



others all contribute to education. National Assessment aoes not 
aim to distinguish one source of achievement from another. 

Moreover, National Assei>sment is not a national examination, 
a set of hurdles which students must pass over for continuing 
opportunity. Neither does National Assessment attempt to measure 
the performance of any one person, school district, or even state, 
nor to award praise or blame to any institution* Although no 
search can be made except from some frame of reference, NAEP 
consistently refrains from interpretive explanations of the data 
collected. Even though perfection is never to be expected nor 
debate cut off. National Assessment is a serious, highly profes- 
sional undertaking. What can be learned from its efforts 
deserves attention. \ 



Consequently, the National Council for the Social Studies 
welcomed the opportunity for independent study, interpretation, 
and dissemination of the assessments in Citizenship and Social 
Studies, two sides of the coin of social studies education. That 
NAEP supported this study with a grant of funds is a mark of 
NAEP professionalism. 

An NCSS Steering Committee took over all responsibility: 
Jean Fair, chairperson, June Gilliard, Dana Kurfman, James 
Shaver, and Ronald Smith. The task of dissemination has been 
the responsibility of the Steering Committee. Five other tasks 



were identified, and major investigators appointed to give time 
and thought to examining them: 1) the assessment model. Bob 
Taylor; 2) methods and procedures, Guy Larkins; 3) the validity 
of the exercises, Francis Hunkins; 4) interpretation of the 
findings, Benjamin Cox; 5) consistency of the exercises with 
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NCSS Social Studies Curriculum Guidelines , desirability, and 
realistically satisfactory performance levels, June Chapin. 
Each of the investigators' final reports was reviewed by the 
Steering Committco and, except for the first, by members of 
task review panels. The first, a delineation of the assess- 
raent model, is accompanied by two commissioned papers speaking 
to the model and prepared by Joseph Grahnis and Michael Scriven. 
Those who worked on this project were not only competent but 
of varying professional roles in social studies education; 
attention was given to men-women ratios, minority groups, and 
geographic areas. The project has aimed for thoughtful, honest, 
and open points of view. \^ 

The Bulletin reports on each of the five tasks of investi- 
gation, but with emphasis upon what has been less fully presented 
elsewhere. Readers will find another emphatsis in a special 
issue of Social Education for May, 1974, on National Assessment 
in Citizenship and Social Studies.^ The compleTte Pinax Reports 
to NAEP will be available through ERIC. Since many people will 
find the publications of NAEP itself useful, a bibliography is 
included at the end of this Bulletin. These Bulletin chapters 
examine the assessments from several stances. Both TayUpr and 
Larkins treat the process of arriving at objectives, methods, 
and procedures, but in differing contexts. Larkins looks at the 
objectives and Chapin *s panel at the exercises with NCSS Guide- 
lines as criteria. Hunkins's panel considered the validity of 
the exercises for the objectives, while Chapin *s panel considered 
the worth of the exercises by the criteria of the NCSS Guidelines . 
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Cox interprets findings from exercises categotizea as social 
studies knowledge by the method of analysis, while Chapin's 
panel judges realistically satisfactory performance levels for 
individual exercises. 

The Project believed National Assessment, as well as 
other assessment programs, better served by stating points of 
vie>^ which have both differences and commonalities. Pinal 

Reports and this Bulletin do attempt, however, to consider 

\.'. 

basic questions: what is assessed, the extent to which the 
assessments can be counted on, what can be learned from the 
procedures themselves, what the findings mean, and the extent 
to which they are useful. 

So ambitious and potentially influential an undertaking as 
9 national assessment of educational progress must from the out- 
set make decisions on a host of issues. Some are highlighted 
here and elsewhere in this Bulletin. 

k number of issues cluster around the matter of objectives. 
Who was to decide, first of all, what was to be assessed in 
citizenship and social studies - or any. other areas? Teachers? 
Administrators? Those in state departments of education or 
those at the grass roots? Experts in social studies education? 
Scholars in relevant disciplines? Textbook authors and pub- 
lishers? Minority groups? Students? Those who support 
custom in education, "what everybody knows it*s always been," 
or those who support innovation, "the cutting edge?" Researchers 
on educational problems? Educational policy makers in legis- 
latures and school boards? 
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Early in the enterprise and after consultation with several 
kinds of people, basic criteria for objectives were set. 
Objectives had to be "D considered important by scholars, b) 
accepted as an educational task by the school # and 3) considered 
desirable by thoughtful lay citizens."^ These criteria pointed 
to the kinds of people who were to decide, if not to the 
particulars of the process. 

Then contracting agencies, in a sense, experts in assess- 
ment, searched the literature of the field out of which came 
a tentative list of objectives. Panels of those competent in 
social studies education or the fields of social science/history, 
teachers and other school persons, and thoughtful lay people 
concerned about education reviewed and revised until a set of 
objectives was formulated. How many kinds of people should be 
heard from? Were a sufficient number of persons from minority 
groups included? Were there too many professional, comparatively 
well-off people, and too few who could see education from the 
vantage of the poor? Were groups who needed information for 
policy-making under-represented? These are matters not readily 
resolvedi especially within the necessary constraints of time 
and cost, nor closed off to future reconsideration. At any 
rate, it is noteworthy that thought from both the pro- 

fessional field and several sorts of people including laymen 
contributed to the formulation . 

Such a process does not make for a theoretically clean and 
consistent set of objectives. On the one hand, it is foolish 
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tp disregard the contributions of scholarship. On the other 
hand# ours is a changing society and a pluralistic one. Not 
even scholars agree. No one official set of objectives exists # 
nor even one which draws wide allegiance « and probably least oi 
all in the area of social studies education. Most of us like 
it that way. It can be argued that no single satisfactory set 
of social studies objectives is either possible or desirable. 
Against the merits of a theoretically consistent conception of 
social studies education must be balanced the need for a set 
of objectives which seem legitimate to many in society. 

Almost from the start assessments in citizenship and social 

studies were separated. Much can be said in favor of two 

\ 

assessments rather than one focused on an area of critical yet 
controversial importance. Many will support the idea that 
citizenship is the responsibility of the school as a whole # not 
merely programs in social studies, or even that citizenship is 
as much the responsibility of out~of-school institutions. But 
if social studies education can be thought of as emphasizing 
what is less likely to be learned informally in the culture at 
large, it can hardly be conceived as something without integral 
relation to individual lives and the requirements of society* 
Nor can citizenship be defensibly conceived as social partici*- 
pation without thought or knowledge. Neither is it sensible to 
think of citizenship as primarily political and social studies 
non^political. Issues of distinction, overlap, and emphasis 
are difficult to resolve. 

Crucial also was the decision about what was to be assessed. 
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National Assessment might have focused only upon some few toasic 
skills, the 3 R's perhaps though they e e far easier to name than 
identify* NAEP might have focused its efforts on assessing 
knowledge and knowledge only in some set of disciplines, or in 
subject areas commonly in school curriculums. NAEP might have 
attempted to assess the outcomes of typical, or pres?umably "best," 
or "poorest" school programs, or, for that matter, out-of school 
educational institutions. The list of possibilities is long. 
The actual decision was for assessing a broad range of fields. 
As a consequence the assessment yields information about aspects 
of educational attainment in the population as a v?hole for which 
data have been sparse. Moreover, the decision throws the weight 
of NAEP to a broad rather than narrow conception of educational 
attainment, especially important in social studies education. 
What is assessed exerts powerful influence on what schools see 
as important to teach and what students see as important to learn. 

Another set of issues is embedded in the closely related 
area of exercises, expected to furnish evidence of attainment of 
the objectives, to be sure, but also to be significant in them- 
selves. When objectives are translated into exercises, the 
chips are down. 

Citizenship and Social Studies are inevitably touchy areas. 
To avoid what is controversial is in itself to take a position. 
To the credit of NAEP, it chose not to rule out the controversial. 
But how much and how sensitive? Review of exercises by panels 
of several sorts of people resulted in rewriting or even dropping 
a substantial proportion of exercises. Social Studies and Citiaen- 
ship were* indeed, more sensitive areas than some others. The 
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issue here is much like that in the matter of objectives. 
What kind of balance can be had between the need for attending 
to the views of many diverse social groups, and the necessity 
for developing exercises legitimate in the eyes of many in the 
country at large? 

One more dilemma appears in the matter of "right answers," 
especially for those exercises about complex problems yielding 
to no simple solution and/or depending upon points of view and 
attitudes. Are all positions taken to be considered proper 
responses if their holders support them with whatever reasons? 
Or must some positions agree with predetermined proper 
responses, for example, support for the rights of the First 
Amendment? Or is some mixture appropriate? Blacks, Native 
American Indians, to name two of a number of groups, experience 
significant differences in their social worlds from those of 
dominant groups. What should be considered proper responses - 
or proper exercises - for such groups? 

Although National Assessment, unlike many school assessment 
programs, has not operated on a. shoestring, it too must function 
within limits of financial support, time, and the capabilities 
of the general field of assessment. NAEP too must make choices 
to what extent are self-reports in exercises justifiable sub- 
stitutes for actual observation of "live" behavior? How much 
effort should be devoted to developing exercises assessing more 
complex, higher cognitive and affective behaviors? And, indeed, 
if such assessment requires much time from respondents, how 
much more time is feasible without throwing the baby out with 
the bath? Could special sub-studies do the job? 
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As does every assessment program, NAEP has hacl hard decisions 
in the construction of exercises. What they developed is a far 
cry, but a heartening one, from what m-^ny people have learned to 
think of as "tests." NAEP exercises in Social Studies and Citizen 
ship frequently resort to paper and pencil, but also rely on 
interviews, and even observations of discussion tasks. If young 
people were asked to respond to the familiar multiple choice 
forms, they were also asked to view pictures; listen to songs; 
use maps, graphs, cards from a library card catalogue, and 
indexes; interact with each other in discussion groups; watch a 
film; and reply to interview questions. If not all, then many 
exercises are lively, innovative, readily related to the present 
social world, and exemplary. \ 

A last set of issues are those of interpreting, the finding. 
Assessment in itself is neither evaluation nor explanation. 

As succeeding rounds of assessments in Citizenship and 
Social Studies are carried on, benchmark data will be available. 
Such comparisons can be made now in Science. "On most exercises 
measuring science knowledge and skills, achievement declined 
at all three school ages assessed — 9, 13, and 17 years" from the 
first to the second assessments.^ However, benchmark data can 
be had now for Citizenship and Social Studies. 

As a guide to interpretation NAEP has developed national 
percentages of successful performance for each exercise (and for 
some sub-groups in the population and categories of exercises) . 
Although illiminating, a few illustrations make it plain that 
these performance levels are not necessarily standards of what 
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is "good," "adequate," or even "bad." When asked in the 
C:itizenship Assessment whether a person on TV or radio should be 
allowed to state any of three generally unpopular views, 
"statements which make some people angry," 3% at age 13, 17% at 
age 17, and 24% of young adults would allow all three statements 
and gave freedom of speech as the reason. Somewhat higher pro- 
portions would allow any one. When asked to identify the meaning 
of monopoly in the Social Studies Assessment, 51% of the 17-year- 
olds and 56% of the young adults, could do so. When askcjd in 
the Social Studies Assessment to read a line graph identifying 
retail prices for eggs and apples over a period of time, 89% of 
13-year-olds, 96% of 17-year-olds, and 91% of young adults were 
able to do so.^ But when the element of interpretation was in- 
cluded in an unreleased Social Studies exercise, fewer - 53% of 
those at age 13, 74% at age 17, and 69% of the young adults - 
could read and interpret a line graph. 

NAEP also compiles results by groups: age, regions, 
sext blacks and whites, parental education, and size and type^ 
of community. For each of these groups differences from the 
national percentages of success are available, and comparisons 
among broad groups possible. Whether these differences are to 
be viewed with alarm or praise depends in part upon the size of 
the difference and the extent to which educational opportunity for 
all is accepted. What is more# if the national percentage of 
success is judged too low, a more successful group performance may 
still be inadequate. 
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The proportion who are able "to do" an exercise depends in 
part upon the difficulty of the exercise. Many standardized 
tests have been set up to distinguish the most able » typically 
able, and least able, and exercises cc.istructed accordingly. 
Although it might have been, such was not the purpose of 
National Assessment. Instead, NAEP aimed to describe the 
educational achievements of the population in four age groups. 
Consequently, NAEP developed about a third each of all exer- 
cises for the least able, typically able, and most able. 
Successful performance percentages must be read and results 
interpreted accordingly. 

Of course, NAEP might have followed still another path: 

c 

hoping for mastery, claiming that everyone should be able to 
perform every exercise successfully. Such a course would have 
required either a set of exercises, within the reach of all, or 
a set of more demanding exercises with a built-in and higher 
"failure" rate. The former would have failed to tap what the 
more able could do* The latter would have loaded the dice, 
emphasized not what has, but what has not been attained. Much 
is to be said for "mastery." Perhaps we have all been too 
tolerant of "not getting it," moving on to something else 
before learning is achieved. The problem, however, comes down 
to agreeing on exactly what, specifically which tasks every 
young person in this broad and diverse land should be able to 
perform. * 

The problems in settling on proper difficulty levels are 
again much like those in agreeing upon objectives. At any 



rate, performance levels over, let us say 90%, can not simply 
ana in themselves be judged as satisfactory, nor those below 
as unsatisfactory, pinpointing areas for improving educational 
endeavor . 

National Assessment is almost inevitably caught between 
the frying pan and the fire. On the one hand, information from 
assessments can be better interpreted and used when tied to a 
school district or a social studies program. There, it seems 
at least, influencing conditions might be sorted out. Better 
yet, research questions and accompanying and flexible inquiry 
designs might point to influencing factors and implications for 
change. The meaning of data comes clearer when explanatory 
matters are related to outcomes, when questions posed give people, 
a handle on the data. On the other hand, fears have been ex- 
pressed from the outset that National Assessment might become 
a national testing program with all its consequent restraints. 
School districts, or states, or those (engaged in some program 
or other - any identifiable persons or groups - could hardly be 
expected to enjoy or seek the glare of examination by some 
outside agency; sometimes it is enough to put up with those 
they can call their own. NAEP has intended to be neither a 
national testing program nor an examining agent. The thought 
of collecting data for identifiable institutions, programs, or 
individuals on a national scale staggers the mind. All that 
means, however, that National Assessifient does not furnish infor- 
mation directly to those making policy decisions for particular 
states, or school districts, or classrooms, or social studies 
programs. 
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Moreover # a national assessment was conceived at a time 
when federal efforts in education were rowing and the spotlight 
on educational attainment in the country as a whole. By now 
efforts have in many respects shifted towards states and 
localities. They have, in turn, their own needs for evidence 
which a national assessment can satisfy only indirectly. De- 
fensible assessment programs are costly, and shoddy or duplicate 
ones unjustifiable, all the more so in times of straightened 
economic cirucmstances. A mere collection of unrelated assess- 
ment data from some states here, some districts there could 
hardly allow for coherence in the whole or for information 
gathered in one place but useful in others. 

In a sense, the strengths of National Assessment - and 
many others at state and local levels - have also been its weak- 
ness. What factors are to explain the findings? What produces 
what? For which policy questions are assessment data to be 
provided? NAEP is now addressing such problems by undertaking 
"special analysis activities requested by USOE to answer 
questions pertinent to federal policy decisions...": for example, 
analysis of results for group combinations, such as race within 
region and community, to provide information on the matter of 
whether "the federal government should devise efforts to redress 
resource imbalance and for whom?'* NAEP has also commissioned 
a study of background factors affecting school achievement with 
an eye toward the feasibility of including some of these in 
NAEP studies.^ Perhaps there are other means by which NAEP can 
include consumers of assessment findings in some ways like those 
of deciding upon objectives. 
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Still some points seem clear and a few cited ftere. 
Typical performance on exercises in Citizenship and Social 
Studies of school-age young people in the Inner City is below 
the nation as a whole; typical performance of young people in 
well-off residential areas is above. Obviously enough large 
proportions of Inner City young people are members of minority 
groups. Whatever can be said in support of arguments that the 
two assessments do not account sufficiently for the experiences 
of sub-cultural groups, it seems plain that attention must be 
paid. Social Studies education and the multitude of conditions 
which influence it have to be better. The complexity of the 
problems does not justify sweeping them under the rug. 

Social studies educators will do well to look at the results 
of specific exercises. Only 41% of 17-year-olds can respond 
properly to all five parts of an exercise on using a simple ballot. 
Nor do young adults pick this up once they become of voting age. 
Surely those in social studies education ought to take steps to 
see that 17-year-olds in their own schools do better on a matter 
so vital. A number of race-related exercises show up a basic 
fund of decency among young people. In the midst of conflict 
and change social studies education ought to find ways to 
capitalize upon it. Some 82% of 13-year-olds and 93% of 17-year- 
olds (and interestingly only 67% of young adults) believe that 
* teen-age students should help decide what courses will be 
offered in their school system. "^^ Such expectations need to be 
accounted for in social studies curricular planning. 

What has been done and what has been found in National 
Assessment in Citizenship and Social Studies is worth thoughtful 
consideration. 
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